update: bump lab pins to ePHPm v0.7.0, split off the removed-engine suites by luthermonson · Pull Request #3 · ephpm/lab

luthermonson · 2026-08-18T22:34:40Z

Bumps the lab's live image pins to the v0.7.0 release.

v0.7.0 publishes the same tag shape as before (release.yml builds
ephpm/ephpm:vX.Y.Z-php<full> + :vX.Y.Z-php<minor> + :<minor>, plus
:vX.Y.Z / :latest for the default minor, which is 8.5). So the lab's
v0.6.3-php8.4 / v0.6.3-php8.5 pins map straight onto v0.7.0-php8.4 /
v0.7.0-php8.5.

The bump is deliberately not uniform

v0.7.0 deleted machinery this lab measures: the rusqlite engine
([db.sqlite] engine = "sqlite" is now a hard startup error), the sqld
sidecar, [db.sqlite.sqld] write_permits, and cdc_experimental. Blanket
-bumping every pin would have produced suites that fail at startup and, worse,
one that would have kept running under a name whose mechanism is gone.

Bumped to v0.7.0 (these still measure what they claim):

What	Files
Kubernetes tier	`k8s/{krayin-v3,laravel-v4,opcache-cluster,php-benchmark,php-benchmark-v2,runtimes-bench,wordpress-v5}.yaml`
DB proxy tier (engine-independent)	`db/bench-proxy.sh`, `db/probe-{reset,pg,clean-vs-dirty}.sh`
Bridge tier (Turso lane)	`db/bench-bridge.sh`, `db/bench-wordpress-bridge.sh`
Driver default	`scripts/run-db-bench.sh`

Hard-pinned to v0.6.3, and made immune to --image / EPHPM_IMAGE:

db/bench-engines.sh (now reads EPHPM_ENGINES_IMAGE)
db/bench-admission.sh (now reads EPHPM_ADMISSION_BASE_IMAGE)

Three of the engines matrix's four lanes are removed machinery. Lane D would
have been worse than merely dead: cluster-turso-primary.toml sets
replication.cdc_experimental = true, ephpm-config does not reject unknown
fields, so on a v0.7.0 image that line is silently ignored and the lane
would have benchmarked a different topology under lane D's label — exactly the
failure mode the suite's own startup-log gates exist to prevent. admission
sweeps a knob that no longer exists at all.

Both are retained as the historical parity evidence behind the engine switch.
Replacing the engines matrix for v0.7.0 means a new Turso-single vs
Turso-CDC-clustered suite, not edits to these lanes — which is what
DB-BENCH.md already said the v0.7.0 bump should do.

Lanes that straddle the release

The rusqlite halves of bridge / wp-bridge are now opt-in
(BRIDGE_LEGACY_SQLITE=1, WP_BRIDGE_LEGACY_SQLITE=1) and hard-run on
v0.6.3. run_lane takes a per-lane image, prints it in the lane banner, and
warns when the two lanes would straddle a release. A v0.6.3 rusqlite lane
next to a v0.7.0 Turso lane differs by a whole release, not by an engine —
they must not share a table.

k8s/runtimes-bench.yaml's bench-ephpm-turso lane is retired at
replicas: 0. It existed purely to A/B the engine knob; on v0.7.0 "turso"
is the only accepted value and the default, so it and bench-ephpm would
select the same engine — the lane would be bench-ephpm wearing a second
label. Scaled to zero rather than deleted so the manifest still records it.

Preserved

Historical prose and every recorded result table are untouched — only the pins
that control what actually runs were changed. The README/DB-BENCH "Historical
(pre-v0.7.0)" blocks were rewritten to describe the new split rather than
deleted.

Also fixes a pre-existing stale claim: k8s/OPCACHE-CLUSTER.md said the
manifests pinned v0.5.0-php8.4 while opcache-cluster.yaml had already
moved to v0.6.3.

Verification

All modified shell scripts pass bash -n.
Remaining v0.6.3-php8 references outside docs are exactly the four
intentional historical pins.
Benchmark numbers are not included in this PR; they are being recorded
separately once the v0.7.0 images finish publishing.

…uites v0.7.0 publishes ephpm/ephpm:vX.Y.Z-php<minor> as before, so the live pins move v0.6.3-php8.4 -> v0.7.0-php8.4 (Kubernetes tier) and v0.6.3-php8.5 -> v0.7.0-php8.5 (single-host database tier). The bump is not uniform, because v0.7.0 deleted machinery this lab measures: the rusqlite engine (engine = "sqlite" is now a hard startup error), the sqld sidecar, [db.sqlite.sqld] write_permits, and cdc_experimental. Bumped to v0.7.0: k8s/{krayin-v3,laravel-v4,opcache-cluster,php-benchmark,php-benchmark-v2, runtimes-bench,wordpress-v5}.yaml db/bench-proxy.sh, db/probe-{reset,pg,clean-vs-dirty}.sh (engine-independent) db/bench-bridge.sh, db/bench-wordpress-bridge.sh (Turso lane) scripts/run-db-bench.sh default image Hard-pinned to v0.6.3 and made immune to --image/EPHPM_IMAGE: db/bench-engines.sh (EPHPM_ENGINES_IMAGE) db/bench-admission.sh (EPHPM_ADMISSION_BASE_IMAGE) Both suites are now historical. Three of the engines matrix's four lanes are removed machinery, and lane D would have been worse than dead: it sets replication.cdc_experimental = true, ephpm-config does not reject unknown fields, so on a v0.7.0 image that line is silently ignored and the lane would benchmark a different topology under lane D's name. Replacing that matrix means a new Turso-single vs Turso-CDC-clustered suite, not edits to these lanes. The admission suite sweeps a knob that no longer exists at all. The rusqlite halves of bridge/wp-bridge get the same treatment: opt-in via BRIDGE_LEGACY_SQLITE=1 / WP_BRIDGE_LEGACY_SQLITE=1, hard-run on v0.6.3, with run_lane now taking a per-lane image that it prints in the lane banner and warns about when the two lanes would straddle a release. A v0.6.3 rusqlite lane next to a v0.7.0 Turso lane differs by a whole release, not an engine. k8s/runtimes-bench.yaml's bench-ephpm-turso lane is retired at replicas: 0. It existed to A/B the engine knob; on v0.7.0 "turso" is the only accepted value and the default, so it and bench-ephpm would select the same engine -- the lane would be bench-ephpm wearing a second label. Kept scaled to zero rather than deleted so the manifest records what it was. Historical prose and recorded result tables are preserved throughout; only the pins that control what actually runs were changed. Also fixes a stale claim in k8s/OPCACHE-CLUSTER.md, which said the manifests pinned v0.5.0-php8.4 while opcache-cluster.yaml had already moved to v0.6.3.

…d engine Missed in the pin bump. bench-proxy.sh drove four lanes off rusqlite: A-lite-inproc single-sqlite.toml A2 / B2 / C2 (via sidecar) litewire-sidecar-sqlite.toml STEP 0 proxy-litewire-inprocess-BROKEN.toml All three configs set engine = "sqlite", so on the new v0.7.0 default they would fail at startup. STEP 0 was the worst of them: it is the deliberately broken lane that demonstrates the proxy/litewire STARTUP-ORDER defect, and on v0.7.0 it would have died of engine validation instead while still archiving its output as FINDING-startup-order.log -- a wrong finding under the right filename. The whole litewire group now runs the Turso engine, which both v0.6.3 and v0.7.0 accept. That is not just a repair: it makes this suite comparable across the bump, because a v0.6.3-vs-v0.7.0 delta on these lanes is now a proxy/pool delta rather than an engine delta. Drops J2-turso-proxy-pool, which ran proxy-litewire-pool.toml against a Turso sidecar while B2 ran the same config against a rusqlite one. With the group standardised on Turso the two are the same lane, and measuring it twice under two names is how a table starts lying.

Recorded on ephpm/ephpm:v0.6.3-php8.5, Windows 11 host, podman machine 32 vCPU / 62 GiB, ephpm container --cpus 1, oha, 8s warmup + 2x15s reps, machine load average 0.00-0.96 throughout. Every bridge cell 100% HTTP 200. bridge: first recording of this suite with this harness, and the last release on which rusqlite-vs-Turso is measurable at all (both lanes on ONE image, differing only in [db.sqlite] engine). Turso barely moves the wire path (-11%..+5%) but moves the bridge path a lot: point-select +38% at c=1 and +43% at c=16. Deleting the wire is worth 1.2-1.8x on rusqlite and 1.3-2.8x on Turso. The ~60x in-process microbench ratio quoted as the hypothesis does NOT survive to the HTTP level -- a full request is mostly PHP and HTTP, not SQL -- and that is now stated where the hypothesis was. proxy: litewire lanes only. The v0.6.1-era shape holds (proxy loses at c=1, pooling wins +104% at c=16), but two integrity problems are recorded rather than smoothed: A2 write c=16 rep 2 returned 1454 HTTP 500s (gate 5: not a measurement), and A2 is unstable at c=1 on writes with a 56% rep-to-rep spread. Cause not established; logged as an open question. FINDING: the STEP 0 startup-order defect no longer reproduces. On v0.6.3 the proxy binds first and resolves its upstream asynchronously ("MySQL proxy listening (upstream connect continues in the background)"), takes one refused attempt, and connects ~250ms later; db.php returns a real SQL error instead of [2002] Connection refused. So ephpm.dev's "Still true in v0.6.1" note is stale, and STEP 0 is no longer the gate that PROVED B2/C2 needed a separate sidecar -- the sidecar is now a deliberate isolation choice. Section retitled accordingly. Also: bench-proxy.sh now checks once for the dbbench-mysql / dbbench-pg upstreams and prints a SKIP with the podman command to start them, instead of letting six lanes each print "!! FIXTURE WRONG -- lane invalid". A missing prerequisite and a failing product must not look the same in a results log.

luthermonson · 2026-08-18T23:20:52Z

Benchmark status: v0.7.0 images never became pullable

This PR bumps the pins, but it has not been validated against a real v0.7.0
image, because no v0.7.0 image exists.

Release run 32190771396
was re-run mid-flight and the v0.7.0 tag was then deleted from the repo
(GET /git/ref/tags/v0.7.0 → 404).
All three Docker (PHP ...) legs consequently failed in
actions/checkout@v4: git fetch ... +refs/tags/v0.7.0* returned exit 1 on
three retries.
Docker Hub confirms it: ephpm/ephpm still tops out at v0.6.3-*. There is
no v0.7.0, v0.7.0-php8.5, v0.7.0-php8.4 or v0.7.0-php8.3.

So every v0.7.0-php8.x pin in this diff is currently a forward reference.
It matches the tag scheme release.yml actually publishes
(:vX.Y.Z-php<full>, :vX.Y.Z-php<minor>, :<minor>, plus :vX.Y.Z/:latest
for the default minor, which is 8.5), so it should resolve once the release is
re-cut — but nothing here has been executed against it. Do not merge until a
v0.7.0 image is pullable and at least the bridge suite has been run green
against it.

What was run, on v0.6.3-php8.5, is in the two follow-up commits: the
bridge and proxy baselines, recorded so the v0.7.0 comparison is a
same-box head-to-head rather than a diff against a stale historical table.

Separately: the tagged v0.7.0 code did not build on Windows

Before the tag was deleted, Build (PHP 8.3.31, windows-x86_64) failed to
compile ephpm-php:

error[E0308]: mismatched types
   --> crates\ephpm-php\src\ws_bridge.rs:357:5
354 | ) -> std::os::raw::c_long {
    |      -------------------- expected `i32` because of return type
357 |     send(id, body, binary != 0).code()
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `i32`, found `i64`

Five E0308s plus one E0277: the trait bound i32: From<u32> is not satisfied.
ws_bridge.rs declares five extern functions returning
std::os::raw::c_long and returns i64 from them. That is fine on Linux/macOS
(LP64, c_long = i64) and broken on Windows (LLP64, c_long = i32), so
the native-WebSocket bridge cannot build on Windows at all. Filed here only as
a heads-up — this is an ephpm/ephpm issue, not a lab one.

…diagnosed Recorded on ephpm/ephpm:v0.7.0-php8.5 (digest c40689f2, image version v0.7.0+php8.5.7, revision c84e3c6). All five expected v0.7.0 tags (-php8.3/8.4/8.5, v0.7.0, latest) verified pullable before measuring. bridge: BOTH arms recorded back to back in one session, because a -20% cross-session delta is exactly what host drift manufactures. The v0.6.3 arm is a fresh control re-run, not the 2026-08-18 recording, and it reproduces that recording on every cell. All 48 cells (24 per arm) were 100% HTTP 200. The result is a clean split: all six bridge cells are flat (-4.9%..+2.2%, at or below this harness's noise), all six wire cells are down 14-31%. Wire and bridge cells run in the same process against the same backend and differ only by the pdo_mysql connect + MySQL frontend, so the cost is in the per-request connect path -- not the Turso engine, not PHP, not the HTTP layer, any of which would have moved both halves. proxy agrees independently: in-process lane A is down 9.9-28.8% while the pooled lane B2 moves 0.9-7.3%, which is what a per-connection cost looks like when a pool amortises it. v0.7.0 is published, so this is shipped: stock pdo_mysql against the embedded engine -- the documented default integration -- is 14-31% slower than v0.6.3 on these fixtures. Applications on ephpm_db_* are unaffected. Hypothesis recorded, NOT established: litewire e34c6392 -> 10345a86 (0.2.0) added per-connection handshake work (ConnectionAuthenticator, random per-connection scramble, TLS feature fence, tenant screen) and turso went 0.7.0 -> 0.7.2. Bisecting the litewire pin against a fixed ePHPm build is the decisive next step. Not done here, and the doc says so. The "bridge is worth 3.7x on v0.7.0" line is flagged as a trap: the bridge did not get faster, its denominator shrank. The ~60x microbench claim still does not survive to HTTP (1.5-3.7x end to end). Also corrected: the 2026-08-18 "load average 0.00-0.96 throughout" note does not survive 30s sampling -- the same harness reaches 5-7 during c=16 cells, because oha is not CPU-capped. Both arms here ran under matched load profiles (median 1.37/1.25, p90 2.57/2.73) with no other workload on the box. FINDING: A2's HTTP 500s are diagnosed and are NOT an ePHPm defect. The defect recurred on v0.7.0 (581x HTTP 500 on write c=16 rep 2, vs 1454x on v0.6.3), which made it worth reproducing under a body-capturing probe rather than leaving as an open question. The error is [2002] "Cannot assign requested address" -- EADDRNOTAVAIL, client-side ephemeral port exhaustion, confirmed by tw 5807 in /proc/net/sockstat against a 28231-port ip_local_port_range. A2 is the only lane opening a fresh remote TCP connection per request with no reuse anywhere. This explains every unexplained feature of those rows: why it is always rep 2 (TIME_WAIT needs ~60s to exhaust the budget), why c=1 writes are bimodal at 42-56% spread (that lane sits exactly at the port boundary), why only A2, and why it is release-independent. A2's four cells are struck from both releases' tables rather than compared. The real-world caveat stands and is stated: a PHP app opening a fresh remote pdo_mysql connection per request hits this ceiling at a few hundred req/s -- which is the cost the bridge and the pool exist to remove. bench-bridge.sh: purge a lane's own files and stamp RUN-INFO before measuring. $OUT persists across runs and the opt-in lane A is usually not run, so its previous session's files sat next to the current run's with nothing to distinguish them -- an archived "v0.7.0" directory here still carried v0.6.3 rusqlite rows. Stale results that look current are the failure these gates exist to prevent.

The load numbers in the previous commit were attributed to the wrong windows. Both load samplers were started for one run and left running into the next, so their aggregates silently mixed the two quiet `bridge` windows with the noisier `proxy` one. Sliced by actual run boundaries: bridge v0.7.0 (21:17-21:24) n=21 median 1.49 p90 1.76 max 1.82 bridge v0.6.3 (21:24-21:32) n=32 median 1.15 p90 1.37 max 1.60 proxy v0.7.0 (21:32-21:45) n=35 median 2.26 p90 3.85 max 7.57 after all runs n=69 median 0.20 p90 4.83 max 13.36 This does not change any RPS measurement, and it strengthens rather than weakens the headline: the two arms carrying the wire regression ran quieter and more tightly matched (max 1.82 vs 1.60 on 32 vCPU) than the previous commit claimed. The 7.57 belongs to `proxy`; the 13.36 is the A2 bug-repro load test and analysis, entirely after measurement. It also means the previous commit's retraction of the 2026-08-18 "load average 0.00-0.96 throughout" note was itself wrong. That claim was rejected on the strength of a mixed-window aggregate; for the `bridge` suite the earlier figure is plausible after all. The retraction is retracted and both errors are recorded in the doc rather than quietly edited out -- attributing a load figure to the wrong window is the same class of mistake as attributing a measurement to the wrong image, which is precisely what these gates exist to catch.

luthermonson added 3 commits August 18, 2026 15:34

luthermonson added 2 commits August 18, 2026 21:53

Navigation Menu

Uh oh!

update: bump lab pins to ePHPm v0.7.0, split off the removed-engine suites - #3

update: bump lab pins to ePHPm v0.7.0, split off the removed-engine suites#3
luthermonson wants to merge 5 commits into
mainfrom
pins-v070

luthermonson commented Aug 18, 2026

Uh oh!

luthermonson commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

luthermonson commented Aug 18, 2026

The bump is deliberately not uniform

Lanes that straddle the release

Preserved

Verification

Uh oh!

luthermonson commented Aug 18, 2026

Benchmark status: v0.7.0 images never became pullable

Separately: the tagged v0.7.0 code did not build on Windows

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant